{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import libraries\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load Excel File\n",
    "filename = 'data/car_financing.xlsx'\n",
    "df = pd.read_excel(filename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Filtering \n",
    "car_filter = df['car_type']=='Toyota Sienna'\n",
    "interest_filter = df['interest_rate']==0.0702\n",
    "df = df.loc[car_filter & interest_filter, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Approach 1 dictionary substitution using rename method\n",
    "df = df.rename(columns={'Starting Balance': 'starting_balance',\n",
    "                        'Interest Paid': 'interest_paid', \n",
    "                        'Principal Paid': 'principal_paid',\n",
    "                        'New Balance': 'new_balance'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Approach 2 list replacement\n",
    "# Only changing Month -> month, but we need to list the rest of the columns\n",
    "df.columns = ['month',\n",
    "              'starting_balance',\n",
    "              'Repayment',\n",
    "              'interest_paid',\n",
    "              'principal_paid',\n",
    "              'new_balance',\n",
    "              'term',\n",
    "              'interest_rate',\n",
    "              'car_type']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Approach 1\n",
    "# This approach allows you to drop multiple columns at a time \n",
    "df = df.drop(columns=['term'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Approach 2 use the del command\n",
    "del df['Repayment']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Identifying Missing Data\n",
    "Values will be originally missing from a dataset or be a product of data manipulation. In pandas, missing values are typically called `NaN` or `None`.\n",
    "\n",
    "Missing data can: \n",
    "* Hint at data collection errors.\n",
    "* Indicate improper conversion or manipulation.\n",
    "* Actually not be considered missing. For some datasets, missing data can be listed as \"zero\", \"false\", \"not applicable\", \"entered an empty string\", among other possibilities. \n",
    "\n",
    "This is an important subject as before you can graph data, you should make sure you aren't trying to graph some missing values as that can cause an error or misinterpretation of the data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Finding Missing Values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two common methods to indicate where values in a DataFrame are missing are `isna` and `isnull`. They are exactly the same methods, but with different names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Notice we have a Pandas Series of True and False values\n",
    "df['interest_paid'].isna().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "interest_missing = df['interest_paid'].isna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Looks at the row that contains the NaN for interest_paid\n",
    "df.loc[interest_missing,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Keep in mind that we can use the not operator (~) to negate the filter\n",
    "# every row that doesn't have a nan is returned.\n",
    "df.loc[~interest_missing,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The code counts the number of missing values\n",
    "# sum() works because Booleans are a subtype of integers. \n",
    "df['interest_paid'].isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "True + False + False "
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
